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This report of the work package two of the QMPP (Quality Management of 
Peer Production in eLearning) project lays the basis for the other work 
packages and piioting work of the project. Thus the key objective of the 
work package is to create an understanding of the various useful methods 
and tools in peer production of eLearning. 

According to the project work pian of the QMPP project, the "objective of 
this work package is to create an understanding of the various useful 
methods and tools in peer production of e-learning. The key activities 
inciude the review the various tools, methods and approaches of peer 
production in e-learning content provision as weil as identification of good 
practices in peer production. The main methods of the work in the work 
package are desk research and coiiection of good practices by various 
partners in their own respective environments." 

The original expected outcomes of the work package were two separate 
reports, namely: 

♦ state-of-the-art report of peer production methods in e-learning 
content provision 

♦ state-of-the-art report of quaiity management approaches of peer- 
produced content. 

However, during the data collection and the authoring process it became 
clear that these two aspects should not be divided into two separate 
reports. Thus in this work package deliverable we have combined these 
approaches. However, the chapter 4 of this report concentrates mainly on 
the quality management approaches of peer- produced content in more 
general terms, as the chapter 5 concentrates more precisely on peer 
production in e-learning. The other chapters are covering both these 
intertwined aspects. 

I n this report we 

♦ introduce and discuss the various areas of peer production and their 



♦ discuss the various tools, methods and approaches of peer production 
in e-learning content 

♦ provision 



potential 
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♦ identify good practices of quality management of peer production in 
eLearning. 

This report has been authored by Mr. Ari-Matti Auvinen (HCI Productions 
Oy) with contributions of the foliowing persons: 

♦ Dr. Ulf-Daniel Ehlers (EFQUEL) 

♦ Ms. Kristiina J okelainen (HCI Productions Oy) 

♦ Mr. David Riley and Mr. Jose Pinzon (lAVANTE) 

♦ Mr. Thomas Fischer, Mr. Thomas Kretschmer and Dr. Walter 
Kugemann (I LI) 

♦ Ms. Michela Moretti (Scienter Itaiy) 

♦ Ms. Laura Fedeli (University of Macerata). 

Many authors have provided case studies and as Annex 1 we have inciuded 
a tabie summarizing the approach of the various case studies. 

Mr. Auvinen has worked as the key author. He has been greatly assisted by 
the work of Dr. Uif-Daniel Ehiers (chapter 5) and the work of Ms. Kristiina 
J okelainen (chapter 6). 
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2 . 

DEFINITION OF THE SCOPE OF THE PROJECT - WHAT 

IS PEER PRODUCTION? 



Peer production can be defined to include the digitai content created, 
edited, enriched by peers, in other words by peopie on the "same 
hierarchical level". The contemporary examples in the Internet of peer 
produced digital content include e.g. YouTube^, Facebook^, blogs, flickr^, 
slashdot.org'^ etc. The various dictionaries define the term "peer" as foliows: 

♦ "a person of equal social standing, rank, age, etc."^ 

♦ "a person of the same rank or standing; a legal equal; a person who is 
equai to another in abilities, qualifications, etc."® 

Often with the term "peer" is also linked the term "peer group", which is 
defined as foilows: 

♦ "a sociai group composed of people of similar age and status"^ 

♦ "a peer group is a group of approximately the same age, social status, 
and interests; generally, people are relatively equal in terms of power 
when they interact with peers"®. 

In the context of involving peers to the educational process, some use also 
the term "peer-to-peer education". Peer-to-peer is often linked in the 
technical sense with the "peer-to-peer networks", which describe mainiy the 
technical linking of several computers with another as equals. Some authors 
claim aiso that the concept of peer-to-peer networks is increasingiy evolving 
to an expanded usage as the reiationai dynamic active in distributed 
networks - not just computer to computer, but human to human. Thus e.g. 
Yochai Benkler claims that associated with peer production are the concepts 
of peer governance and peer property in the digitai world.® 

For our purposes it is needed to note that peer-to-peer eLearning often 
refers to such instances, in which the peer learners within an educational or 
training institution are developing eLearning contents to each other for 
limited use^°. 



^ see http://www.voutube.com 
^ see http://www.facebook.com 
^ see http://www.flickr.com 
see http://slashdot.org 

^ see http://www.thefreedictionarv.com/peer + (read 16 lulv 2008) 

® see Webster's Encyclopedic Unabridged Dictionary of the English Language, Random House 1989 
^ see http://www.thefreedictionarv.com/peer+qroup (read 16 lulv 2008) 

® see http://en.wikipedia.org/wiki/Peer group (read 16 lulv 2008) 

^ of Benkler, see Benkler, Yochai: The Wealth of Networks. Yale University Press, USA 2006. 

see e.g. Kotzinos, D. et al.: Online Curriculum on the Semantic Web: 

The CSD-UoC Portal for Peer-to-Peer E-learning. Proceeding of WWW 2005, May 10-14, 2005, Chiba, 
J apan. 
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However, in this QMPP project our aim is also to utilize the peer produced 
content also outside of one specific educational setting. 

The term of "peer production" in the Internet context has similarities with 
the term "user-created content". User-created content has no widely 
accepted single definition (see e.g. OECD's study on User-created content, 
2007), but according to the guidelines by the OECD in their study, in this 
research paper user-created content is defined with three criteria, which are 
the following: 

♦ content is made publicly available over the Internet 

♦ it refiects a "certain amount of creative effort" 

♦ it is "created outside of professionai routines and practices". 



As a conclusion, in our QMPP project, we want to emphasize the 
experiences of peer production, which includes also the strong presence of 
the peer group in the various phases of the learning provision. According to 
our reading of the iiterature, the user-created content is mainly describing 
the digital artefacts produced by various individuals, as in our approach the 
communicative element is essential - and it takes often place by the strong 
involvement of the peer group. Thus according to the different terms and 
their use we can summarize them in the foilowing picture (see picture 1). 

Picture 1 
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OECD - Working Party on Information Technology, 2007: Participative Web: User-created content. 
DSTI/ICCP/IE(2006)7/FINAL OECD 
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The group emphasis is also highlighted in the discussion of "informal 
learning" - e.g. Jay Cross states that informal learning is strongly fueled by 
the communication of peers and that this communication is the critical 
element in informal learning. 



Cross, J: Informal Learning: Rediscovering the Naturai Pathways That Inspire innovation and 
Performance. Pfeiffer 2006. 
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3 . 

THE GROWI NG I MPORTANCE OF PEER PRODUCTI ON 



The potential peer production and user-created content has been supported 
by the deveiopment of Internet from one-way information distribution to 
two-way communication - the Internet browsers with their various plug-ins 
have been becoming essential tools for wide distributed content creation. 
Many schoiars, such as Yochai Benkier, claim that ordinary users wiil enter 
the nucleus of digital content production. 

In particular, these novel opportunities have been discussed in conjunction 
with the deveiopment of "Web 2.0", which has opened up new opportunities 
also for peer production, participation of a wide amount of individuals to 
various discussions and development projects, as well as flexible ways to 
personalize digitai content. The peer production mechanism has been 
central e.g. in the development of Linux software and the Wikipedia 
movement. The new working forms and their consequences have been 
discussed in more detail e.g. by Don Tapscott and Anthony Williams. 

The peer produced content can also reach through the Internet new users. 
Thus it can be also described as the "iong taii" of learning provision - i.e. 
providing also such content elements, which are not used by a wide 
audience, but which might be critically important for certain target groups. 
The "long tail" idea and its opportunity for endless demand has been 
discussed in more detaii by Chris Anderson^^. 

An elementary discussion linked with the potential of the peer production 
has been sparkled also by the work of James Surowiecki and his notion of 
the "wisdom of the crowds"^®. Surowiecki as weil as Charies Leadbeater^^ 
have discussed the theme how "many are wiser than few" and how this 
knowledge creation and knowledge assessment can be distributed and thus 
also made more precise and efficient. However, in the contemporary 
discussion the widely utilized peer production has also been criticized in 
promoting "the cult of the amateurs" and thus promoting the content 
creation of amateurs without appropriate quality approach. This discussion 
has been initiated in particular by Andrew Keen.^® 



Benkier, Yochai: The Wealth of Networks. Yale University Press, USA 2006 
Tapscott, Don - Williams, Anthony D.: Wikinomics. Portfolio, USA 2006 

Anderson, C: The Long Tail : Why the Future of Business Is Selling Less of More. Hyperion 2006. 
Surowiecki J : The Wisdom of Crowds: Why the Many Are Smarter Than the Few. Abacus 2005. 
Leadbeater, C: We-think: The Power of Mass Creativity. Profile Books 2008. 

Keen, A: The Cult of the Amateur: How Today's Internet Is Killing Our Culture and Assaulting Our 
Economy. Nicholas Brealey Publishing 2007. 




Education and Culture 



8 





QAfPP 



WP2 - Quality and Evaluation 
Setting the scene - 
introduction to quality in peer production of eLearning 



Furthermore, in modern management literature the importance of co- 
creation experiences and their importance to the value creation has been 
emphasized in particular by C.K.Prahalad and Venkat Ramaswamy/® Their 
main argument is that the market is becoming a forum for conversation and 
interactions, and that the management and facilitation of this dialogue is 
the key in value creation process. Their view of the market means that the 
market is becoming instead of a seller- buyer- market rather the arena for 
co-creation of value. Furthermore, they define that the key building blocks 
for the interaction between users and providers of services can be defined 
as 

♦ dialogue 

♦ access to important information and resources 

♦ risk- benefit assessment by the users 

♦ transparency of work and working methods. 

The aspect of "dialogue" is the essential element in peer production and 
user-created content. It is elementary that the dialogue focuses on issues 
that interest both the users and the providers; that there is a forum in 
which the dialogue can occur; and that there are clear rules of engagement 
that make for an orderly, productive interaction. The web-based 
environments provide usually such a "forum", where not only the users and 
the providers can meet, but also the users can meet with another. 
C.K.Prahalad and M.S.Krishnan develop this idea further in claiming that the 
co-creation of value can also occur on the global level, and that the 
cocreated experiences can be even more personalized. 

Thus it can be concluded that the facilitation of peer involvement is seen as 
one elementary factor in the overall improvement of learning and training 
provision. However, regarding the quality aspects and quality mechanisms 
of peer production it can be seen that relatively little attention has been 
paid to the quality assurance of this new production mechanism of 
eLearning. 



Prahalad, C.K. - Ramaswamy, V: The Future of Competition - Co-Creating Unique Value with 
Customers. 

Harvard Business School Press 2004. 

Prahalad, C.K. - Krishnan, M.S.: The New Age of Innovation - Driving Cocreated Value through 
Global 

Networks. McGraw-Hill 2008. 
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4. 

PEER PRODUCTION IN PRACTICE IN THE 
VOCATI ONAL CONTEXT 



This chapter describes the peer production mechanism in two different 
areas: nameiy in education and training as well as in other related 
vocational areas. In undertaking the authoring of this report, it became 
obvious for the authors (mainly Mr. Ari-Matti Auvinen and Ms. Kristiina 
Jokelainen) that there was documented a very limited amount of 
experiences of peer production in the area of vocational education and 
training. However, in the areas of higher education there were some 
interesting articies as weli as in the area of primary and secondary 
education. However, the key term was often "user created content” - not 
clearly "peer production". 

The impact of peers in vocationai training and practice has not emerged just 
recently. Many methods have been implemented in various workplaces and 
vocational environments for long, such methods include, among others, the 
following: 

♦ work shadowing 

♦ mentoring 

♦ coaching 

♦ Job rotation 

♦ communities of practice and organized knowledge sharing. 



4.1 Peer production in knowledge creation and 

KNOWLEDGE MANAGEMENT 

In the vocational setting, the one of the cornerstones has been the deeper 
understanding how knowledge should be managed. The views expressed 
more than ten years ago by Ikujiro Nonaka and Hirotaka Takeuchi in their 
work on the knowledge creation in companies, and in particuiar, on the 
models and patterns how to alter personal knowledge to corporate 
knowledge, and how to collect tacit knowledge and convert tacit knowledge 
to explicit knowledge, are essential also in organzing and utilizing peer 
production. 



of the various practices, see e.g. American Society for Training and Development: State of the 
Industry Report 2007. 

Nonaka, Ikujiro - Takeuchi, Hirotaka: The Knowledge-Creating Company. Oxford University Press, 
USA 1995. 
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In our context, the essential challenge for peer production eLearning is to 
accelerate the transformation of personal knowledge towards corporate or 
common knowledge. 

The essential element in successful knowledge management is that the 
creation and utilization of knowledge is not limited to some key individuals, 
but is rather seen as an elementary part of the work of all co-workers. 
However, it is also important to highlight that effective knowledge 
management can be an important - if not the most important element - in 
creating learning resources within a company. 

In particular, the opportunities have been discussed in conjunction with the 
development of "Web 2.0”, which has opened up novel opportunities also 
for peer production, participation of a wide amount of individuals to various 
discussions and development projects, as well as flexible ways to 
personalize digital content. In particular, two authors have been describing 
the new opportunities in a interesting and valid way, namely Don Tapscott^"^ 
and Yochai Benkler.^^ 

In addition to the general description of the novel opportunities of the "Web 
2.0" and its various tools and methods, one of the essential aspects to be 
discussed in linked with the ownership of jointly created content, which will 
be an important issue to solve in any organization.^® 



Case study: Eureka at Xerox^^ 

Eureka project was started within the Xerox company in 1994. The 
challenge for Xerox was to create a working mechanism to collect and 
distribute technical and maintenance information for the front-line people. 
This was based to the notion, that within the Xerox company the majority of 
technical knowledge was "inside the heads of the field people" - in other 
words not in well- structured databases or documents. 

The principal idea in the Eureka project was to support the field engineers 
and technicians to share in their Xerox community maintenance advice, tips 
and "good practices". In addition. Xerox faced two important challenges, 
which were the following: 



on this, see in more detail Ahmed, P.K. - Kok, L. K. - Loh, A.Y.E.: Learning through knowledge 
management. Butterworth Heinemann 2002. 

Tapscott, Don - Williams, Anthony D.: Wikinomics. Portfolio, USA 2006. 

Benkler, Yochai: The Wealth of Networks. Yale University Press, USA 2006. 

on the ownership aspects, see in particular: Ghosh, Rishab Aiyer: CODE - Collaborative Ownership 
and the Digital Economy. The MIT Press, USA 2005 and Hietanen, Herkko - Oksanen, Ville - 
Valimaki, Mikko: Community Created Content. Turre Legal, Helsinki 2007 

sources for this case study included, among others, Ahmed, P.K. - Kok, L.K. - Loh, A.Y.E.: Learning 
through knowledge management. Butterworth Heinemann 2002 and Brown J.S. - Duguid P.: 
Balancing act: How to capture knowledge without killing it. Harvard Business Review 78 (3) (2000), 
73-80 
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♦ the maintenance manuals were outdated quickly and the printed 
versions were soon out-of-date and difficult to maintain in an accurate 
condition 

♦ the maintenance manuals did not include many innovative front-line 
work methods or practices, but they documented rather clinically a 
"clinical" work practice. 

The experiences Xerox collected of its front-line co-workers showed that, in 
addition to the official technical documentation - and in many cases instead 
of the official technical documentation - the front-line people used their own 
notepads, selected only certain important pages of the manuals and wrote 
their own procedural notes. A part of the unofficial practices was the "peer 
copying" (or if you like: xeroxing) and "peer distribution" of the self-made 
and self-summarized technical documents. 

Within the Eureka project a simple-to-use database was constructed and 
every front-line person could contribute to that database and save his/her 
own experiences. The basis structure for the input was the following: 

♦ product/ component/ part 

♦ experienced problem 

♦ proposed solution 

♦ chain of messages (tree metaphor of messages - e.g. a new message, 
answer, comment etc.) 

♦ author 

♦ validator. 

The database could be accessed also through a search engine, which was 
either a drop-down menu or free text search. The tools developed within the 
Eureka project included also a peer review tool (close to what e.g. Amazon 
uses today in peer rating of their books), by which the users could assess 
the usability and validity of each input. It was also important for the 
company, that the Eureka documents were routed to the R&D entities, 
which could thus receive direct, real, and timely feedback from the front- 
line. 

Eureka provided its users also with small guides (hint books) covering the 
following subjects: 

♦ authoring - what to cover, how to describe the problem and solution 
etc. 

♦ validation - how to validate, what aspects to consider, what references 
to use etc. 

♦ sharing - how to share my knowledge, what are the key benefits of 
sharing etc. 

♦ utilization - how to use, what are the limitations of using Eureka etc. 
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Although the reward system of producing content to Eureka was one critical 
aspect to consider, in some countries the front-line co-workers spoke 
against monetary rewards of information to be saved into Eureka, as they 
claimed it might corrupt the quality of the content, would it only be 
provided triggered by monetary rewards. 

As a summary one can state that by using the Eureka Xerox increased its 
confidence to their own front-line organization, their problem-solving skills 
and their continuous innovativeness. In addition. Xerox claims that 
according to its own calculations, it has saved 100 million USD (by the year 
2000) in various stages of its operations by using the Eureka approach. 



The Eureka case shows also, that the conventional authoring and utilization 
of service and maintenance documentation of modern technical equipments 
and systems (directed to the front-line operations) are in many aspects 
ineffective activities. This is caused by a number of various reasons, such 
as: 

♦ the maintenance documentation is based on "just- in-case" thinking 
instead of "Just-in-time" thinking ; a great amount of manuals and 
documents are authored, validated, translated and distributed 
although practically there is no evidence, whether these manuals and 
documents are ever used in the actual front-line (field) operations 

♦ the key documentation can be described as "clinical documentation" 
instead of "experimental documentation" ; as the content creators are 
often not linked with the actual field organization and their practical 
day-to-day work, the actual utilization context and its prerequisites are 
not visible in the documentation 

♦ the creation mechanism of technical front-line documentation is based 
rather on "documentation experts" than on "front-line experts" ; a 
majority of the documentation is either duplicating already existing 
documentation, but the documentation can also be too granular for 
front-line use 

♦ the decreasing life-cycle of technical equipment (and, in particular, 
some key components or elements of software) means also often that 
before the full cycle of the technical documentation is undertaken 
(from creation through validation, translation, distribution, training 
etc.), actually the documentation is already outdated. 
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However, the most important result in summarizing the content to the most 
critical one and to have people from the actual front-line organization to 
develop these materials, is the economy of labor in daily work situations. As 
often noted, engineers seek to learn what is needed to do their job; there is 
no time to learn things completely. Thus the real issue in workplaces is to 
cope effectively with real-life situations. 

The case of Eureka in Xerox shows how the peers in their daily working 
environment can collect and document valuable information and refine that 
further into knowledge. Many areas of professional life have their own 
mechanisms in promoting creation and distribution of important knowledge 
elements; in some cases - like in the medical profession - this knowledge 
also passes organizational boundaries and is targeted to the wider 
professional audiences. 



Case study: Bank of innovative practices at the lAVANTE 

foundation^^ 

Bank of Innovative Practices (BPI) has been developed by lAVANTE 
foundation for the Ministry of Health in the Andalucian Regional 
Government. It is a platform with the aim of fostering and improving 
innovation and diffusion in the Andalucian health sector, by means of 
identification of innovative actions which will be carried out in itself and its 
diffusion to the rest of the system, so that they can act from pilot or 
exemplary experiences to the rest of the System. Each innovative practice 
or innovative idea is developed by a peer professional group from the 
system. 

BPI allows the registration as participant, which offers the following 
alternatives that can ease peer production: 

♦ to inform and to be informed about the innovative practices which are 
being developed in the Andalucian Public Health System 

♦ to create collaborative working spaces for the development of 
innovative practices between professionals of different organizations 
(belonging to the Ministry of Health for the Andalucian Regional 
Government) 

♦ to communicate with other professionals or working parties with 
similar problems 

♦ to value and to give one's opinion about innovative practices which are 
being developed, as well as to contribute with ideas and suggestions. 

The BPI provides its participants with a set of collaborative tools that allow 
for the peer production of content. Project teams have at their disposal a 



among numerous sources, see e.g. Reardon, Robert F.: Informal learning after organizational 
change. The Journal of Workplace Learning, Vol 17, nr. 7 (2004), pp. 385 - 395 
this case study has been authored by Jose Pinzon (lAVANTE) 
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common space where they can share documents, post links, comments and 
calendar events. In addition to that, there are other site-wide toois avaiiable 
such as a tagging system for ail content and a voting system. Comments by 
users are allowed for every content item in the site. Users can choose the 
configuration of their public profile. 

The registration in innovative practices which have recognition as such after 
the evaluation by BPI, entitles members of the team responsible of them to 
get a certificate. Moreover, BPI prizes are offered, pursuing the recognition 
of professionals' task and the repetition of those initiatives that are more 
innovative. 

For each innovative practice (a project) or innovative idea (a proposai), 
there’s a set of parameters analyzed by the Bank’s Steering Committee. 
Projects are evaluated on aspects such as efficiency, effectiveness, level of 
innovation, gender equality, and adequacy with wider heaith poiicies. For 
the ideas or proposals, innovation potentiai, feasibiiity and potential impact 
are assessed. 



However, the potentiai of peer production is not oniy iimited to the 
knowledge creation and distribution by the professionals. For instance in the 
health care sector, Internet is growing in its importance as a critical 
resource provider for other non-professional users. In many cases, the 
critical amount of content in the healthcare area is created by peers - 
providing their experiences, hints, examples, and providing also space for 
empathy and camaraderie. Numerous studies show also that an important 
proportion of the users utilise - in addition to the "ciinicai content” provided 
by the official healthcare sector - also these user-created Internet 
resources.^® 

Furthermore, the close interaction with the users of healthcare is also one 
part in the new value creation metaphor in various industries, including 
healthcare. Thus it is claimed that the value creation mechanisms are 
moving towards ’’co-creation of value". 



see e.g. Houston, T.K. - Ehrenberger, H. E.: 'The Potential of Consumer Health Informatics". 
Seminars in Oncology Nursing 17 (2001) (1), 41 - 47 and Demiris, G.: "The diffusion of virtual 
communities in healthcare: Concepts and challenges". Patient Education and Counselling 62 (2006), 
178 - 188. 

on this see e.g. Auvinen, A-M., 2007: "Personal Health Systems and Value Creation Mechanisms in 
Occupational Healthcare". Engineering in Medicine and Biology Society, 2007. EMBS 2007. 29th 
Annual International Conference of the IEEE, pp. 5882-5885. 
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It is also interesting to note that the corporate field has taken new 
technologies and solutions in peer production on board differently. The 
recent Finnish study showed that the wikis are increasing in various 
companies as an important method to document and distribute important 
knowledge. However, in the virtual worlds the experience is negative - 
according to the studies of Gartner Research, the vast majority of virtual 
world projects launched by businesses fail within 18 months. In addition, 
fully 90% of business forays into virtual worlds fail because organizations 
focus on the technology rather than on understanding the needs of the 
employees using it.^^ 



Case study: Wiki for Health Care Professionals (~ 'Pflege-Wiki')^^ 

The Wiki for Health Care Professionals (i.e. 'Pfiege-Wiki') is a multilingual 
(German and English) Wiki project for Health Care Professionals and 
especially for nurses. The Wiki (German version^^) started in August 2004 
as a private initiative and is run since J uly 2005 by the 'Association for the 
Promotion of Unrestricted Information on Nursing’ ("Verein zur Forderung 
freier I nformationen in der Pfiege e.V.”). This association finances the 
services via member fees and donations. The English version^® was added in 
May 2006; in March 2008, collaboration with the Italian NursePedia^^ was 
established. 

At the moment, the German version contains more than 4.400 articles free 
for use and distribution in the sections: 

♦ health care in practice (with subchapters like geriatric care, sick- 
nursing, pediatric nursing, etc.) 

♦ terminology - pathology 

♦ vocational education and training 

♦ management & pedagogy for the health care sector 

♦ academic studies 

♦ professional associations; press releases 

♦ links, forums, chat 

♦ care facilities, ambulant care services 

♦ book reviews, events, podcast. 



Henriksson, J. - Mikkonen, T.: Wiki-kokemuksia suomalaisissa organisaatioissa (Experiences of Wikis 
in Finnish organizations). Tampereen yliopiston hyperme(dialaboratorio. Hyperme(dian 
verkkojulkaisuja 18, 2008. 

reporte(d in I nformation Week at 

http:// WWW. informationweek. com/ news/ personal_ tech/ virtualworl(ds/showArticle.ihtnnl? articlel D =207 
800625 (read 17 J uly 2008) 

this case study has been authored by Thomas Fischer and Thomas Kretschmer (I LI ) 
see http://www.pfleqewiki.de 
see http://en.nursinqwiki.org 
see http://www.nursepedia.net 
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The 'Pflege-Wiki' - like typical Wikis - offers the possibility to read, write and 
modify articles about subjects relevant to the health care sector. Each 
contribution is reviewed by the other users of the service, so that each 
article is produced collaboratively. 

Like other wikis, 'Pflege-Wiki' records former versions of articles in order to 
make the history of an article / a concept visible. In rare cases it is used to 
re-establish an article after intended or unintended 'vandalism'. 

The 'Pflege-Wiki" animates visitors not only to read, write and modify the 
articles, but also to add photos, to link the Wiki to personal and official 
websites and to distribute the articles in other media (e.g. printed). All 
material within the 'Pflege-Wiki' is published under the GNU General Public 
License and can be used and copied without any restrictions^®. 

Each article as well as each registered user have a discussion forum, where 
authors and users can get in contact or clarify certain aspects of an article. 
Registration is not necessary, but offers certain advantages (e.g. each 
article is signed with the nickname of the author; registered users have a 
personal page for self- presentation; registered user can 'observe' certain 
articles and receive a notification in case of modifications of these observed 
pages). 

'Pflege-Wiki' offers a chat for all visitors of the wiki as well as podcasts 
(usually interviews with experts concerning actual problems and 
discussions). 



4.2 Peer production in vocational training and 

EDUCATI ON 

If peer production is an inspiring method in the joint knowledge creation 
and knowledge refinement, it also has a lot of potential in the area of 
vocation education and training. The conventional approach to organized 
training at the workplaces is often in the human resources function (which 
report to the Vice-President of Human Resources) there is also a separate 
training unit or training department. This training department employs 
trainers, who often are also the persons in charge of the internal production 
of learning materials.®® 

Although it is highlighted that the training function should actively have a 
dialogue with the mainstream employees of the company / organization, 
often this is taking place only by means of running training needs analysis’, 
organizing courses, collecting feedback etc. 



for more (details please see http://en.wikipe(dia.orq/wiki/GNU General Public License 

of the typical training structure, see e.g. Wills, M.: Managing the training process - Putting the basics 

into practice. McGraw-Hill 1993. 
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As many authors have claimed - and Jay Cross with his concept of "informal 
learning" probably with the sharpest edge - the companies and 
organizations have hugely underutilized in their training and learning 
activities the potential of the employees with practical experience and thus 
created learning resources and materials, which do not correspond to the 
needs and style of the users. In addition, in many organizations much 
additional work is undertaken due to this "doubling" and "re-authoring" of 
the same content. 



Case study: KONE Elevators and peer- produced technical training^" 

The model of "workplace-centered learning material production" has been 
implemented in various organizations - among others, the Alcan aluminum 
factory in Kurra Kurra (Australia) documented their approach in a sound 
and condensed manner. In a pilot project in KONE Elevators (the large 
Finnish elevator company) Australia, new models to create front-line 
maintenance information by the front-line people themselves were piloted. 

The important basic notion was that in developing maintenance instructions 
and manuals for the front-line, the quality of the content can be improved 
by using as an author an experienced supervisor than a documentation 
expert or a trainer. This was simply due to the fact that the experienced 
supervisor knew and could govern the variety of existing front-line 
documentation, and thus avoid double or triple amount of work in creating 
the documents (thus e.g. in an maintenance documentation the general 
process of a maintenance operation was only described once, if it was 
different from the ordinary maintenance operation; if it was similar to the 
ordinary maintenance operation, there was Just a link created to the 
conventional procedure). Another key element was that this method also 
condensed the front-line documentation, as it was concentrating on the 
essentials and thus it was also shorter than the "just-in-case 
documentation". 

The key component of this model was thus the utilization of the supervisors 
in both content creation as well as content validation. They were supported 
by a number of critical tools (such as resource guides). The experience in 
the project showed, however, that the supervisors could more accurately 
than the documentation unit of the HQ identify the key essential guide lines 
and shortcuts - and thus they could also avoid unnecessary work phases 
and authoring of unnecessary (or overlapping) documents. 



this case study is authored by Ari-Matti Auvinen (HCI Productions Oy) based on the first hand 
personal experiences in working with KONE Elevators 
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The results showed also that the front-line initiated technicai documentation 
was by far cheaper in its globai costs (authoring, transiation, printing, 
distribution) than centrally created documentation was. However, the basis 
of the meaningful work load in the front-line and thus the cost savings is 
the appropriate division-of-labor: the various pianning documentation (e.g. 
CAD pictures etc.) must be done centraliy, but the front-line experts are far 
more efficient in summarizing the created knowledge to field-friendly 
documentation and can effectively link the new documents to the existing 
mass of documents. 

It could also be added, that in many cases in actual front-line maintenance 
work, the fieid engineers and technicians have as a specific problem the 
variety of equipment stemming from different manufacturers - e.g. nearly 
40 % of the lifts KONE Elevators is maintaining are manufactured by 
another iift manufacturer or a singie paper factory might have numerous 
providers of technical equipment. The documentation of the various 
manufacturers is seldom structured in a similar way, and the existing 
documentation does not cover the issues of compatability etc.). 

However, the most important result in summarizing the content to the most 
critical one and to have people from the actual front-line organization to 
deveiop these materiais, is the economy of iabor in daily work situations. As 
often noted, engineers seek to learn what is needed to do their job; there is 
no time to learn things completely. Thus the real issue in workplaces is to 
cope effectiveiy with real-iife situations. 



The previous case of KONE Elevators shows also that there often is poor 
iinkages between the technicai documentation function and the training 
function even in companies and organizations, which are orientated to 
technical maintenance and their field operations. 

The discussion of the "learning organizations" which started in fact by the 
breakthrough book of Peter Senge in 1990"^^, has been stressing that 
learning is becoming an essential part of everyday work. However, it is also 
highlighted that learning is a critical element in deveioping organizational 
cohesion through continuous learning. Also in the deveiopment of the 
learning organization, it was seen as essential to continuously learn for 
others and distribute positive learning experiences within the organization. 
The discussion of the learning organizations aiso emphasized - in the spirit 
of knowledge management - the importance of empowering the various 
learners also to contribute with their personal impact the learning and the 
deveiopment of various iearning resources. 



see Senge, P.M.: The Fifth Discipline - The Art & Practive of The Learning Orgnization. Century 
Business 1990. 

see e.g. Marquardt, M.J.: Building the Learning Organization. ASTD & McGraw-Hill 1996. 
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Case: The Finnish Tax Academy^^ 

The Finnish Tax Academy is in charge of the learning activities within the 
Finnish Tax Administration. The challenges in the work of the employees 
within the Tax Administration include, among others, the following: 

♦ quickly changing regulatory environment (and thus permanent need 
for updates and maintenance) 

♦ expertise in taxation issues really is within the Tax Administration 
operational staff (and thus restricted potential to utilize external 
expertise). 

The Finnish Tax Academy has been an active user of eLearning since the 
year 2000 - in 2007 approx. 1300 employees studied at least on one 
eLearning course (more than 20 % of the workforce). The key of the 
eLearning approach is that all the eLearning courses are produced by the 
Tax Administration staff and experts; within the staff of approx. 6000 
persons, there are more than 100 trained eLearning content providers and 
110 eLearning tutors. 

Authoring eLearning courses provides an important potential to enrichen the 
work of many experts by adding the learning provision task to their work. 
This is enabled by a clear workload assignment and thus eLearning content 
creation is an elementary part of the expert's work; for instance, an expert 
can use 32 person-days per course unit (20 learner hours) of allocated 
workload. It is also important that instead of compensating separately for 
the learning material provision, it is seen (in the spirit of a learning 
organization) as an organic part of everyday work- 

The peer producers of eLearning content have been speciaiiy trained by an 
intensive training program, which has been organized by an external 
university entity. It consists of eight face-to-face training days as well as of 
own web- based courses and own web- based platform for communication, 
discussion and reflection. 

The process of eLearning course content authoring has been jointly defined 
to include the following steps: 

♦ first edition 

♦ peer reviews - "cross auditing" 

♦ revised edition 

♦ piioting with a iimited audience (including structured feedback 
collection) 



this case study is authored by Ari-Matti Auvinen (HCI Productions Oy) based on the first hand 
personal experiences in working with the Finnish Tax Academy, see also the Annual Report of the 
Finnish Tax Administration at http:// www.vero.fi/nc/doc/download.asp7id =6169: 1863506 (read on 
18 July 2008) 
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♦ final version 

Operational and pedagogic support is provided both by the own Tax 
Academy team as well as by external actors (universities, polytechnics, 
consultants). 

As a result of this work, the Tax Administration regards eLearning as an 
elementary part of their learning, the Tax Academy produces or reviews 
more than 20 eLearning courses annually (in all they have more than 60 
course on offer) and they also provide their eLearning courses to all Tax 
Administration employees with a free access also for self-studying. 



The new opportunities for peer in different organizations has been created 
by the various professional online communities known as "communities of 
practice". Although already previously the professionals of different areas 
have been networking and exchanging their experiences, the Internet has 
opened novel avenues for sharing experiences and insights. As Moore also 
points out, networking is taking place both between the organizations as 
well as within the organizations among peers. 



Case: ANITEL - National association of e-learning teachers/ tutors^^ 

It was born in 2004 in order to provide answers and solutions to the 
problems met by teachers playing the role of e-tutor. It has been organized 
directly from teachers’ perceived general need to share inputs, ideas and 
resources in any step of the instructional process. The association has been 
officially recognized: teachers who enroll and successfully complete a course 
are awarded a certification recognized by the Ministry of Education . 

The association is designed to motivate teachers through an active 
exchange of experience and expertise, it plays the role of a "community of 
practice" for teachers in every field (hard science and social science) and 
covering any level of instruction (from primary school to secondary school). 

ANITEL organizes different courses during each year, teachers can enroll for 
free and get their attendance certificate at the end; each course can 
differently take advantage of the following platforms (LMS, CMS and various 
interaction tools): 

♦ public portal to accomplish an informative function (at 
http://www.anitel.it/anitel/modules/news/ ) 

♦ private platform for teachers' courses (at http://fad.anitel.it/ ) 



Moore, M.G.: Network Systems: The Emerging Organizational Paradigm. The American Journal of 
Distance Education, (2003) 17 (1), 1-5. 

this case study is authored by Laura Fedeli (University of Macerata) based on personal experiences 
and involvement 
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♦ maiiing iist using 


yahoo 


groups 


(at 


htto:// it. arouDS.vahoo.com/arouD/Soci 


Anitei/) 






♦ bioa (at httD://bioa.anitei.ora) 








♦ iand in Second Life (97, 239, 401). 









4.3 Peer producti on of new soluti ons and programs 

The peer production as a production mechanism has been chaiienging the 
conventionai production metaphor in some interesting areas. The most weii- 
known exampie the deveiopment of Linux based on the open-source 
ideoiogy. As Tapscott et ai. note, the creation of Linux iiiustrates how the 
internet has enabied many contributors - aii acting independentiy in their 
own seif- interest, create a highiy integrated "good” that provides vaiue for 
themseives and to others."^® 

The "open-source movement” has aiso been getting stronger and as 
Goidman and Gabriei point out, the interest in using open-source code 
residing in the commons by corporations is strong enough that some are 
starting to adapt their internai product iife cycie and deveiopment 
methodoiogies to accommodate the nature of open source. 



Case study: Sourceforge^^ 

SourceForge.net is the worid's iargest Open Source software deveiopment 
web site, hosting more than 100,000 projects and over 1,000,000 
registered users with a centraiized resource for managing projects, issues, 
communications, and code. 

SourceForge.net has the iargest repository of Open Source code and 
appiications avaiiabie on the internet, and hosts more Open Source 
deveiopment products than any other site or network woridwide. The 
essence of the Open Source deveiopment modei is the rapid creation of 
soiutions within an open, coiiaborative environment. 

Members of the Sourceforge community are responsibie for deveioping the 
software that drives the web such as Apache, the worid's most popuiar web 
server, and it is the toois provided by Sourceforge that they use in their 
work. 



Tapsoctt, D. - Ticoll, D. - Lowy, A.: Digital Capital - Harnessing the Power of Business Webs. Nicholas 
Bearley Publishing 2000. 

Goldman, R. - Gabriel, R.P.: Innovation Happens Elsewhere - Open Source as Business Strategy. 
Elsevier 2008 

this case study has been authored by Michela Moretti (Scienter Italy) based on the materials by 
Marcus Clements (Brighton Art, United Kingom) within the framework of Praxis project 
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A project at Sourceforge may be started by a developer or a group with a 
software project at one of several stages of development 

♦ an idea for a Software project 

♦ a proposal with a specification and development plan 

♦ a functioning or partially functioning software application 

♦ a fully functioning software title 

In most cases the initiating developer wants to attract other developers to 
help with the project and has chosen Sourceforge as the best place to do it. 
The vast majority of projects are started by inexperienced developers, 
attract little or no interest from others and after a brief flurry of activity, 
languish in the archives until the initiator finally loses interest. At the other 
end of the scale a very few are extremely successful, with many of 
participants and thousands of downloads of the projects products by the 
public. 

Sourceforge provides statistics about the projects it hosts with an overall 
'activity rating’ that is calculated from the overall popularity of the project 
amongst its participants and by the public. 

Sourceforge consists of a large number of projects. Anybody can start a 
project and begin using the tools provided. There is no charge for the 
service; it is funded by advertising, sponsorship and user donations. When a 
project is started it is immediately provided with a complete set of tools to 
begin work. Each project on Sourceforge has the same set of tools. 

During the development of a software project, a great deal of existing 
knowledge is utilised and new knowledge created. All software is built upon 
a foundation of existing software so a comprehensive understanding of the 
connecting modules and system within it operates is vital for a project to 
succeed. 

Types of knowledge utilised or created in a software project: 

♦ technical documentation of related or connected software 

♦ books & articles describing techniques or processes 

♦ archived discussions from previous related projects 

♦ source code 

♦ help files & FAQs 

Everyone understands the value of archiving information. We only need to 
be asked the same question twice before it's a good idea to write down the 
answer somewhere and make it easily found by the next person with that 
question. 
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Thus elementary in Sourceforge are the archived discussions (neariy ail 
important decisions made by developers collaborating on software projects 
will be made in some kind of archived discussion, be it archived email, a 
forum or a news group) and documentation (as a project continues the 
consensus of opinion and accumulated knowledge that is represented in the 
archives is compiied into a more formai structure in the project 
documentation). However, also cross-referencing and linking are essential 
elements in the successfui projects. 

The success of Sourceforge shows clearly both the demand in the open- 
source area for such a joint workspace, which enables communication 
among peers in often very compiex issues. However, also the provision of 
effective tools for all various actors within the Sourceforge is one critical 
element of success. 



The "Wikipedia movement" has been an interesting peer production 
exercise, as it has been able also to revolutionize the wide participation into 
the creation of joint content. The growing number of contributions into 
Wikipedia (authoring and editing of articles) has created a virtuous circie, in 
which more contributors mean more interaction in terms of more articles 
and therefore, more editing that will attract more contributors. Thus, as 
Rask concludes, there is a linear relationship in Wikipedia between "reach" 
and "richness". 



Case study: Wikipedia^" 

According to their own presentation on their web site (see in more detail 
http://www.wikipedia.org ) "Wikipedia is a multilingual, Web-based, free 
content encyclopedia project. The name Wikipedia is a portmanteau 
(combination of words and their meanings) of the words wiki (a type of 
collaborative Web site) and encyclopedia. Wikipedia's articles provide links 
to guide the user to related pages with additional information. 

Wikipedia is written collaboratively by volunteers from all around the world. 
Since its creation in 2001, Wikipedia has grown rapidly into one of the 
iargest reference Web sites, attracting at ieast 684 miliion visitors yearly by 
2008. There are more than 75,000 active contributors working on more 
than 10,000,000 articles in more than 250 languages. 



Rask, Morten, "The Richness and Reach of Wikinomics: Is the Free Web-Based Encyclopedia 
Wikipe(dia Only for the Rich Countries?" . Proceedings of the Joint Conference of The International 
Society of Marketing Development and the Macromarketing Society, June 2-5, 2007 (available at 
SSRN: http ://ssrn.com/abstract=996 1 5 8 ) i 

this case study has been authored by Ari-Matti Auvinen (HCI Productions Oy) and is based on the 
Wikipedia site ( http://www.wikipedia.org - read on 17 July 2008), on Tapscott, D. - Williams, A.: 
Wikinomics. Portfolio, USA 2006, and on personal experiences 
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In July 2008, there are 2,460,544 articles in English; every day hundreds of 
thousands of visitors from around the world make tens of thousands of edits 
and create thousands of new articles to enhance the knowledge held by the 
Wikipedia encyclopedia. 

Visitors do not need specialized qualifications to contribute, since their 
primary role is to write articles that cover existing knowledge; this means 
that people of all ages and cultural and social backgrounds can write 
Wikipedia articles. Most of the articles can be edited by anyone with access 
to the Internet, simply by clicking the edit this page link. Anyone is welcome 
to add information, cross-references or citations, as long as they do so 
within Wikipedia's editing policies and to an appropriate standard. 
Substandard or disputed information is subject to removal. Users need not 
worry about accidentally damaging Wikipedia when adding or improving 
information, as other editors are always around to advise or correct obvious 
errors, and Wikipedia's software is carefully designed to allow easy reversal 
of editorial mistakes.” 

Wikipedia is a new social innovation, and by using peer production there has 
been created a new form of encyclopedia to be used in the Web. However, 
as the number of articles and contributors has grown, also Wikipedia has 
had to alter its operative mechanisms, including the creating norms on 
authoring and editing the articles, developing the internal quality criteria 
and quality processes and modifications in its user administration. In the 
various articles, Wikipedia also has warnings of the potential incompleteness 
of articles, missing references etc. 

The key issue in Wikipedia has been the eagerness of various people to 
share their knowledge and information with other users. The "social 
discipline" within Wikipedia has been strong and the huge majority of 
authors of Wikipedia are experts in their own area. Wikipedia's strength 
has also been the opportunity to comment and edit the content, and thus 
also to add comments from different perspectives. 

In admiring the success of Wikipedia, it should also be stressed that 
solutions, such as Wikipedia, require also strong centralized resources - 
these are e.g. data structure, content structure, structure of quoting, search 
tools and search structure, various tools to author and edit content etc. 
Fluent dispersed content creation requires a strong centralized structure, 
which can enable various dispersed operations. 



According to the quality of the content of Wikipedia, there has been a lively 
discussion of the trust in the Wikipedia content and also comparisons of the 



Wikipedia content vis-a-vis the more 
McGuiness et al. claim, trust is a central 
and environments that use information 
sources. 



conventional encyclopaedia. As 
issue when dealing with systems 
coming from multiple, unknown 
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Thus it is essentiai, how one can use trust information to heip users view 
and filter information in collaborative and evolving information repositories 
such as Wikipedia. Furthermore, they propose that in digital collaborative 
environments, such as Wikipedia, trust can be divided into citation- based 
trust and revision history-based trust.^^ 

Furthermore, the social impact of Wikipedia has also been discussed widely. 
Although some authors - like Don Tapscott - claim that Wikipedia will also 
remarkably narrow the "digital divide", Rask claim that also in Wikipedia 
countries with a higher level of human development have a competitive 
advantage over the countries with a lower level. He also concludes that the 
level of human development is crucial in participating in sharing knowledge 
at a global scale. Based on this notion it is naturally interesting to discuss, 
whether the various organizations thus also vary in their work with the 
wikis. 



4.4 The different contexts and models of peer 

PRODUCTI ON 

Based on the discussion in this section as well as the cases presented, we 
can present a simple classification of the various critical dimensions of peer 
production in eLearning and related fields. The two critical dimensions are 

♦ setting of objectives - who is setting the objectives for the peer- 
produced content: is it controlled and unified for all potential peer 
producers (e.g. a framework of a "course") or is it relatively loose (e.g. 
in Wikipedia authoring an article of my personal interest) 

♦ structure - what structure is given: is the structure firm (e.g. 
composed of certain elements, tools and requirements) or the 
structure loose and not pre-defined (e.g. peers can use tools and 
structure of their own choice). 



McGuinness, D.L. et al..: Investigations into Trust for Collaborative Information Repositories: A 
Wikipe(dia Case Stu(dy. WWW2006 Workshop on the Mo(dels of Trust for the Web (MTW'06), 
E(dinburgh, Scotlan(d. 
see Rask 2007 
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This is summarized in picture 2. 



Picture 2 
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5 . 

QUALITY APPROACHES TO PEER PRODUCTION IN 

ELEARNING 



The quality of eLearning has been discussed and researched in many 
European projects as well as in many international contexts. Ehlers and 
Pawlowski describe, in the discussion on quality of eLearning, one can 
distinguish between three different aspects in the discussion, namely 

♦ different interpretations of quality 

♦ different stakeholders with different perspectives on quality 

♦ different forms of quality (input-quality, process-quality, output- 
quality). 

This discussion has also lead to different interpretations of quality - and 
numerous definitions from various fields are available. As in particular the 
wide discussion on quality in eLearning in general is well captured by the 
recent comprehensive book edited by Ehlers and Pawlowski^'^, in this 
context it is not necessary to repeat this discussion, but rather concentrate 
on the key issues which are altering in the peer production of eLearning. 

In the peer production of eLearning the essential feature is that the learners 
are also acting as creators of the content - in the new learning settings the 
separation between an "author" and a "consumer" is blurring. In practice, 
learners are no longer purely consumers but they actively participate in the 
learning process and thus influence it. As the borders between user and 
author are blurring, so do the roles of student and teacher. 

This fundamental feature is also imposing a different view on quality, as 
quality is often to be defined and assessed by the same group of actors as 
the actual creation of the learning content. However, the quality approach 
to peer production can be more than Just an emphasis on self-evaluation 
and its practices. 

Many quality approaches also in eLearning rely on the conventional quality 
cycle. This quality cycle has included - since the writings of W. Edwards 
Doming in the 1950s - the steps of PDCA (Plan, Do, Check, Act). This 
approach has been modified during the last decades in many different ways, 
and also applied into the area of vocational training. 



Ehlers, U-D. - Pawlowski, J.: Quality in European e-learning: An introduction. In Ehlers, U-D. - 
Pawlowski, J. (eds.): Handbook on Quality and Standardisation in E-Learning. Springer 2006. 
see Ehlers, U-D. - Pawlowski, J. (eds.): Handbook on Quality and Standardisation in E-Learning. 
Springer 2006. 
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Wirth has presented that in eLearning the essential steps could be: 

♦ plan 

♦ do 

♦ check 

♦ compare. 

As a hypothesis we can claim, that in the development of a quality approach 
to peer production, the quality approach would mainly address the "process 
quality" issues - in other words: which processes implemented are assisting 
the quality of peer produced eLearning content. In addition, in regarding the 
quality processes, the peer production of the learning materials in their 
quality assessment can also be linked with benchmarking - or even more 
precisely, on "benchlearning". 



5.1 Quality development of peer production^® 



Ehlers discusses the quality development in what he is referring to as 
"eLearning 2.0". In describing the phenomenon "eLearning 2.0", he points 
out that it describes a number of developments, trends and points of view, 
which require change from teaching to learning. The new point of view 
essentially connects e-learning with five characteristics: 

1. Learning takes places always and everywhere (ubiquitous) and 
therefore in many different contexts, not only in the classroom. 

2. Learners take on the role of organizers. 

3. Learning is a life-long process, has many episodes and is not (only) 
linked to educational institutions. 

4. Learning takes place in communities of learning (so called communities 
of practice)^^. Learners participate in formal, as well as informal 
communities. 

5. Learning is informal and non-formal, takes place at home, at the work 
place and during leisure time and is no longer centered on teachers or 
institutions. 



Wirth, M.A.: An analysis of international quality management approaches in e-learning: Different 
paths, similar pursuits. In Ehlers, U-D. - Pawlowski, J. (ecds.): Han(dbook on Quality and 
Stan(dar(disation in E-Learning. Springer 2006. 

this chapter is based largely on the writings of Ulf-Daniel Ehlers and his article "Web 2.0 - eLearning 
2.0 - Quality 2.0 - Perspectives on a change in learning culture and quality concepts" . In 
Hohenstain, A. - Wilbers, K. (eds.): Handbuch E-Learning. Koln 2008. 

on communities of practice, see e.g. Wenger, E.: Communities of practice: learning, meaning, and 
identity. Cambridge University Press, 1999. 
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In the new environment, learners are highly self-directed, as learning does 
not only take place in institutions, but everywhere, during the course of 
one's whole life in a number of different episodes, in learning communities 
and social networks, using social software and individually compiled 
contents. Securing and developing quality in such learning scenarios thus 
has to focus mainly on the individual learning processes and the shown 
achievements (performance). The learner's perspective is more important 
than the organizational processes and / or the co called input factors. 
Quality assessment does not take place by using classical methods of 
expert- and standard based quality management, quality assurance or 
control, but by making use of more participative methods and responsive 
designs. The aim of the process is to reach an individualized assessment, 
which relates to the learning process. 

Initially it seems paradox to talk about the quality of eLearning 2.0, as 
quality is often linked with checking by externally imposed standards. 
However, quality can also be understood in a development-oriented way, 
which means the enabling of learners to develop themselves in their own 
learning processes and consequently reach better results as far as quality is 
concerned. In this view, methods of self-evaluation, reflection and peer- 
evaluation are seen as more important. This kind of quality methodology 
does not have anything to do with normative, universally valid, but aims at 
improving the quality of the learning process. 

In eLearning 2.0 learning approach, the learner has an important role as 
active constructor of learning materials (co-creator), personal learning 
environments and initiator of his or her own learning processes. 
Interestingly, this is a characteristic, which is often felt to be a barrier for 
integrating eLearning 2.0 into formal educational processes. This is because 
the competition of learners and teachers and/or other institutional actors 
during quality assessment seems to be insurmountable and only resolvable 
through a loss of power for the institution. 



5.2 Conditions for quality of peer productions^ 

As a point of departure, eLearning 2.0 does not require a new mode of 
thinking or method of quality development, such as a new and completely 
altered philosophy of quality - no "educational quality 2.0" is needed. 
However, changed basic conditions and contexts need to be taken into 
account. Doing justice to these different contexts, different questions need 
to be posed when dealing with quality development, different objects 
evaluated, different criteria of quality applied and specific methods of 
quality assurance, enhancement and development used. In short: the role 
of quality development is changing. 



this chapter is based largely on the writings of Ulf-Daniel Ehlers and his article "Web 2.0 - eLearning 
2.0 - Quality 2.0 - Perspectives on a change in learning culture and quality concepts”. In 
Hohenstain, A. - Wilbers, K. (eds.): Handbuch E-Learning. Koln 2008. 
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While in traditional learning scenarios it mostly means the checking and 
controlling of quality, in eLearning 2.0 it is becoming more the role of an 
enabler of learning progress. Learning methods and quality development 
are moving closer together. Methods such as feedback, reflection and 
recommendation mechanisms are becoming more important. Typical basic 
conditions, which need to be taken into account in quality development for 
eLearning 2.0 scenarios, are explained in the following: 

♦ From reception to participation: the metaphor used for learning is 
changing. In eLearning 2.0, quality cannot be tied to the evaluation of 
a pre-determined learning environment or learning contents produced 
by an expert. Not the reception but the active participation is most 
important, that means the question in how far a learning scenario 
stimulates the creation of individual personal learning environments, 
the compilation of individual learning contents and sharing them with 
others. 

♦ From inspection to reflection: quality development for eLearning 2.0 
shifts the focus from conformity to a reflection of the learning process. 
Learners are supported in reflecting, recognizing and putting into effect 
their own learning progress, educational strategies, needs, etc. and in 
the course of their actions critically reflect the contribution of 
educational media. The aim is to achieve a personally ideal 
configuration of educational media and strategies, which is 
continuously developed through autonomous reflection. 

♦ From product orientation through process orientation to performance 
and competence orientation: the material that is used for learning and 
the processes of its supplier are not the focus of quality development. 
Quality development focuses on the learners' performance, their 
individually developed learning products, steps in development and 
similar aspects (for example in e- portfolios), which shape their way to 
decision-making and responsibility. 

♦ From planning education for the leaner to planning education by the 
learner: quality of learning scenarios is often attempted to be achieved 
through careful analysis of the need for education, a comprehensive 
conception phase, feedback as far as the design of learning material 
and development processes are concerned and the evaluation of 
learning processes. In eLearning 2.0, many of these processes shift 
from the supplier of a program to the learner. Quality concepts must 
therefore support the learners in their ability to develop quality 
through reflection, enable learner-oriented forms of evaluation and 
offer the necessary tools for quality development to the learners in 
their PLEs. 
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♦ From receiver to developer of learning materials: quality assessment in 
eLearning 2.0 scenarios does not follow the logic of a marketing 
effectiveness research to find out how the materials and characteristics 
of media optimally affect the learning process. It is not about learning 
process taking part in a unified learning scenario. Rather, the focus lies 
on processes of development, flexible usage and the validation of 
social communication processes with other learners. 

♦ From the "learning island" LMS to the internet as a learning 

environment: eLearning 2.0 approach understand Learning 

Management Systems (LMS) as a mere starting point, as a signpost for 
their own search and use of material from the internet, their 
development and linking to other tools which can be flexibly arranged 
to become personal learning portals. Quality assessment then does not 
focus on materials from the LMS anymore but rather on the learning 
products and perhaps on the learning processes documented in an e- 
portfolio. 

♦ From tests to performance: learning progress and achievements 
become visible not only in tests but rather in the learning process 
documented in portfolios (for example in wikis or web logs), learning 
products and social interactions. 



5.3 Key concepts for the implementation of quality in 

PEER PRODUCTION^® 

Quality assessment of eLearning 2.0 focuses on the learning process. There 
is no use of external standards and inter-individual comparisons (such as 
tests or assessments). Rather, methods of self-evaluation, intra-individual 
development processes are employed for this purpose, which are not made 
via tests but via reflection and evaluation of learning products and e- 
portfolios. Even though eLearning 2.0 is a new development as a trend, 
substantial experiences have already been made with the learning models 
of autonomous learning and learning in communities, which are the basis 
for it, as well as with methods for quality assessment of learning processes. 

In particular, three concepts are worth discussing in more detail, namely 
self-evaluation, e-portfolios, and social recommendation by peers. 

The concept of self-evaluation includes enormous potential for quality 
assessment of learning processes in eLearning 2.0. The aim of it is not a 
complete (summative) assessment of learning achievement, but rather an 
improvement of learning abilities. 



this chapter is based largely on the writings of Ulf-Daniel Ehlers and his article "Web 2.0 - eLearning 
2.0 - Quality 2.0 - Perspectives on a change in learning culture and quality concepts” . In 
Hohenstain, A. - Wilbers, K. (eds.): Handbuch E-Learning. Koln 2008. 



Education and Culture 



32 



QAfPP 



WP2 - Quality and Evaluation 
Setting the scene - 
introduction to quality in peer production of eLearning 



In scientific literature, positive effects of self-evaluating processes on the 
learning achievements can be found®°. When undertaking these processes, 
students can gain insights into the profile of their own strengths and 
weaknesses. It has also been shown, that if students evaiuate their own 
achievements positively, they aim for more chalienging objectives, engage 
in their own learning process more and mobilize more personal resources®^. 

Web-based portfolios (e-portfoiios) integrate different media and services. 
Learners collect desired learning products in their e-portfolio, which are 
made in the course of a class or even during the whole course of their 
studies. Students can use eiectronic portfoiio to show competences and 
refiect their learning processes. Learning results, connected with remarks 
by tutors, teachers and peers, feedbacks and personal reflections are 
coliected. E-portfoiios iend themselves also to quality assessment. Learning 
scenarios supported by e- portfolios emphasize the learning process and 
enable a deeper understanding of learning processes by all participants. 

In eLearning 2.0 communication, feedback and the exchange within a 
learning communities is essential. With the help of social software tools 
collaborations can be conducted and information exchanged, as well as 
evaluated mutually. Three methods are of special significance and first 
experiences have been made: 

♦ social recommendation mechanisms 

♦ peer review method 

♦ peer assist method. 

Social recommendation mechanisms are defined as those methods that 
serve the purpose of assessing the "true quality" of learning material®^, in 
contrast to methods focused on experts. According to this method, the 
members of a learning community evaluate materials available online. On 
the one hand this method can be understood as "quality evaluation" in the 
course of which each learning material is assessed by learners. On the other 
hand it is also possible to give learners recommendations - a la Amazon - 
on which learning material is thought to be especially useful, so called social 
recommendations. 



“ see e.g. Maehr, M. - Stallings, R.: Freedom from external evaluation. Child Development 43 (1972), 
177-185 and Hughes, M. - Ribbins, P. - Thomas, H.: Managing Education: the system and the 
institution. London 1985. 

Rolheiser, C. - Ross, J.A.: Student self-evaluation: What research says and what practice shows. In 
Small, R.D. - Thomas, A. (eds.): Plain Talk About Kids. Covington 2001. 

Duval, E.: LearnRank: Towards a real quality measure for Learning. In Ehlers, U.-D. - Pawlowski, 
J.M.: Handbook of Quality and Standardisation in E-Learning. Heidelberg 2006. 
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Peer review is a concept that has been introduced a number of times, 
especially in the academic sector. It deals with assessing quality by peers - 
that is colleagues or other learners - giving each other feedback. In the 
area of learning, especially in eLearning 2.0 settings, the peer review can be 
used to attain feedback and quality assurance for resuits, iearning progress 
and aims, which is given from other learners or members of the learning 
community. Peer reflection is a process aimed at creating situations for 
rejecting, in which the peers are asked to encourage the reflection of 
learning processes by means of their own experiences. 

One possibility to check on the quality of learning processes is iearning from 
other peopie's solution, respectively entering a peer learning process with 
others. One model that has recently been gaining more importance is the 
peer assist model. It is a structured reflection in the context of a sociai 
network, which is carried out via social software. This method is cieariy 
distinct from peer review, as its primary aim is to simulate learning 
processes. By employing the method for eLearning 2.0 scenarios, social 
assets are used for further developing one’s own solutions or for resolving 
learning difficulties, which come up in the learning process. Structured 
reflection of a learning process is possibly by broaching the issue of the 
iearning processes, the resuits and documented outcomes in the peer assist 
process. 
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6 . 

TECHNI CAL TOOLS AND TECHNOLOGI ES FOR PEER 

PRODUCTION 



In this chapter we summarize the technical tools and the technology which 
are used in peer production. It is also necessary to define some key terms 
which are closely related into the peer production concept. Although during 
the recent years learning and information provision through Internet and its 
different services have become one of the most popular theme in 
educational development, it is good to keep to remember both the relatively 
short history of the "Internet age” as well as the utilization of some of the 
key concepts already earlier in different contexts. 



6.1 Technical TOOLS AND TECHNOLOGIES in the "Web 2 . 0 " 
ERA 

The key to the potential to use Internet and its services in vocational 
education and training stems from the fact that the metaphor for using 
Internet has changed from information distribution and delivery (which 
many call the "Web 1.0") into active participation and information creation 
by the users (which may call the "Web 2.0"). In particular, with the 
introduction of called "social media" and "social networks"®"^ the peer 
production has been recognised, not only as a challenge the traditional 
education, but also creating a new valuable approach into education - and 
particularly into the eLearning development. 

Actually the term of "Web 2.0" was first introduced 2005 by Tim O'Reilly®^. 
Explaining "Web 2.0" is not that simple - it should be pointed out that 
"Web 2.0" is not any new remarkable invention which has been the starting 
point for new innovations in web- based tool development. Perhaps it is 
better to say that it is more like common name showing all that intensive 
development what has happened in web-based communication. Thus it is 
rather an evidence for achievement of a next stage in networking and 
communication over the Internet. The "Web 2.0" can be seen as a concept 
which sums up different development trends of web- based services, 
applications standards, technologies and most of all how these all are used 
in social communication and networking. 



however, it is needed to be noted that the discussion of "sociomedia" in its eariy years stems already 
from the beginning of the 1990s - see e.g. Barrett, E. (e(d.): Sociome(dia - Multime(dia, Hyperme(dia 
and the Social Construction of Knowledge. The MIT Press 1994. 

on early stages of social networks - see .e.g. Harasim. L. et al.: Learning Networks. The MIT Press 
1995. 

O'Reilly, T.: What is Web 2.0? - Design Patterns and Business Models for the Next Generation of 
Software - http://www.oreillv.de/artikel/web20.html (read 24 J uly 2008) 
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In education and training development and implementing eLearning we also 
can see claim that there is a move towards "eLearning 2.0” in which 
different web communication and information flow tools are used in 
educational and learning purposes. However, the (open) social networks 
and social media are critical for peer production and its integration to 
traditional learning approaches. We understand that "social media" is 
describing the media environment, which enables mutual participation, 
information production and receiving, handling and providing it for others to 
use. As discussed in previous chapters, the social media have been essential 
in the success of user-created content We see the (open) "social networks” 
as a prerequisite for successful peer production. 

There are basically following user-created content, which are then 
distributed through different platforms, technical tools in peer production. 

One important feature in the recent development is also that the creative 
power of the users has been nurtured by the provision of different media for 
peer production. Thus the users are not limited to text only - they can also 
produce easily and cheaply their content by other media, such as photos, 
images, audio clips, video clips etc. Another important feature is also that in 
addition to the conventional computer networks, also novel networks - such 
as mobile networks - can be widely used. 



6.2 Techni cal tools for peer producti on 

In the following we describe some contemporary approaches for peer 
production - these might often be called also Web 2.0” tools. However, new 
tools and approaches emerge daily and these which are novel or upgraded 
versions about those one used commonly today - or totally new services. 
Although many tools used also in the "Web 2.0” environment have also 
their commercial dimension, the important feature in the development of 
many new "Web 2.0” tools is their open source background. These tools can 
be also seen as services, applications or distribution platforms, which 
provide platforms for peer production. With the open source approach also 
the business models and earning models alter - however, open source does 
not mean free or totally non-commercial.®® 

Blogs appear already since midl990s when they were mainly presented as 
personal diaries on various Internet home pages (the name stems from the 
combination of web -i- log book). Today the blogs have turned to be - 
among others - efficient tools to express opinions, to distribute information 
and also to serve as commonly read personal diaries. Modern blogs can also 
easily mix different media - text, graphics, pictures, video clips, audio clips 
etc. 



see e.g. Gol(dman, R. - Gabriel, R.P.: Innovation Happens Elsewhere - Open Source as Business 
Strategy. Elsevier 2005. 
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What makes blogging so powerful is the linking of different "bloggers" in 
blogging platforms by forming so called blogospheres where information can 
spread extremely fast. The linking of teh different blogs has become a very 
important tool for peer production particularly in cases where students 
themselves are encourage to become peer producers and at the same time 
being active in commenting blogs of each other®^ The blogs are often 
tagged with pointing out the keywords or listed according the titles which 
are then visible in different blogosphere or blog account hosting sites - also 
typical for blogging is regular updating which will often "push" old 
information to be invisible and history of the blogging is thus difficult to 
follow. 

Typical for blogging tools is that most of them are free for the users and 
thus the threshold to start active blogging is very low.®® 

Wikis and other text based coilaboration formats are describing a 
webpage or set of web pages that can be easily edited anyone who is 
allowed access - the most well-known of the wikis is Wikipedia^®. The 
fundamental idea of wikis is to provide information voluntarily, 
decentralised and openly. The information can be added, corrected or totally 
new topics can be created without changing the whole structure of the site. 
Information providers are also reviewers of the information. Wikis provide 
also a way to common structured knowledge creation and distribution and 
thus they can serve as an effective tool for learning. 

The quality of information has been staying rather solid in the various wikis 
- despite of minor vandalism appearing time to time in wiki-based web 
sites. Today is easy to create own wiki environment for various topics. 
There are so called "wiki farms" for hosting the sites, which have 
encouraged development of wikis also in universities and other educational 
institutions. In wikis the history of information or individual page is always 
visible and that helps to keep provided information valuable. 



Tagging and social bookmarking allow users to save their bookmarks 
online, tag them and share them with others. In practice it is possible to 
install from web (free of charge) programs which help you to store your 
bookmarks online, tag them and share them with your colleagues and 
students. Use of tagging has become common way to look for information 
by using keywords. 



on 6(ducational uses of blogs - see e.g. Williams, J.B.: Exploring the use of blogs as learning spaces 
in the higher education sector. Australasian Journal of Educational Technology 2004, 20(2), 232-247. 
see also Richardson, W.: Blogs, Wikis, Podcasts, and Other Powerful Web Tools for Classrooms. 
Corwin Press 2006. 

see e.g. http://c4lpt.co.uk/Directorv/Tools/bloqqinq.html (read 24 J uly 2008) of the various blogging 
tools available 

see http://www.wikipedia.org 

see e.g. Parker, K.R. - Chao, J.T.: Wiki as a Teaching Tool. I nterdisciplinary J ournal of Knowledge 
and Learning Objects, 2007 (vol 3), pp. 57 - 72. 

see e.g. http://c4lpt.co.uk/Directorv/Tools/wiki.html (read 24 July 2008) of the various wiki tools 
available 
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By using the tagging the user is abie to select information wanted. The key 
to the applications of social bookmarking is metadata (information on 
information), which enables the sharing of this information. The probably 
best known social bookmarking application is dei.icio.us, in which in which 
users can tag each of their bookmarks with a number of freely chosen 
keywords. 

Different toois used for tagging and social bookmarking heip users to search 
and identify information with keywords and topics and to save it all in own 
directory with favorites/ bookmarks online. In practice they are personal 
websites where user can store archive their favorite web pages. The 
importance of tagging and social bookmarking is in sharing the important 
links and information sources with other users. This enables - instead of 
replicating the information content on certain web sites - the sharing of the 
links and ensure the validity of the information.^^ 



Media sharing means the services which enabie everyone interested to 
share multimedia in the public. Most common example of this service is 
YouTube for videos^^. There are plenty of different sharing services for 
videos, photos and podcasting. 

Podcasting as a term were initiated with launching of iPod - a portable 
digital audio piayer by Appie. Contemporary, the term is referring any 
software and hardware combination that permits automatic downloading of 
audio files for listening at the user's convenience. The key breakthrough for 
podcasting has been the audio signal compression technology - in 
particuiar, the MP3 standard. 

In education the podcasting is a powerful approach as the students are 
familiar with the underlying technoiogy application. By combining the audio 
biogging with podcasting the iectures, experts’ interviews etc., it is possibie 
to provide convenient methods in delivering the educational content. Also 
the students can easily add their own content using audio blogging and 
distribute those audio biogs by podcasting. 

Podcasting is becoming increasingly popular in education. Podcasts enable 
students and teachers to share information with anyone at any time. An 
absent student can download the podcast of the recorded lesson. It can be 
a tool for teachers or administrators to communicate curriculum, 
assignments and other information with parents and the community. 
Remarkabiy, a number of leading US universities and colieges provide their 
selected lectures through the iTunes University site hosted by Apple. 



see http://(del.icio.us 

see e.g. http://c4lpt.co.uk/Directory/Tools/bookmarking.html (read 24 July 2008) of the various 
social bookmarking and tagging tools available 
see http://www.youtube.com 

see http://www.apple.com/education/itunesu mobilelearninq/ itunesu.html (read 24 J uly 2008) 
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Newer WEB 2.0 services 

The is in development a number of new services, which will also change the 
environment of peer production in eLearning. This following summary table 
is based on the publication by J I SC Technology and Standards Watch from 
February 2007 and it is authored by Paul Anderson. We have added the 
last three areas from other sources to the table by Anderson. 



Social networking 


Professional and social networking sites that 
facilitate meeting people, finding like minds, sharing 
content - uses ideas from harnessing the power of 
the crowd, networks effect and individual 
production/user generated content. 


Aggregating 

services 


Gather information from diverse sources across the 
Web and publish in one place. Includes news and 
RSS feed aggregators and tools that create a single 
webpages with all your feeds and emails in one 
place - uses idea from individual production/user 
generated content. 

Collect and aggregate user data, user 'attention' 
(what you look at) and intentions - uses ideas from 
the architecture of participation, data on epic scale 
and power of the crowd 


Data "mash-ups" 


Web services that pull together data from different 
sources to create a new service (i.e. aggregation 
and recommendation). Uses, for example, ideas 
from data on epic scale and openness of data- 


Tracking and 

filtering content 


Services keep track of, filter, analyse and allow 
search of the growing amount of Web 2.0 content 
from blogs, multimedia sharing services etc. Uses 
ideas from e.g. data on epic scale. 


Collaboration 


Collaborative reference works (like Wikipedia) that 
are built using wiki- like software tools. Uses ideas 
from harnessing the power of the crowd. 

Collaborative, Web- based project and work group 
productivity tools. Uses architecture of participation. 


Replicable office- 
style software in 
the browser 


Web-based desktop application/document tools. 
Replicate desktop application. Based on 

technological development. 


Sources ideas or 
work from the 
crowd 


Seek ideas, solutions to problems or get tasks 
completed by outsourcing to users of the Web. Uses 
the idea of power of the crowd. 



Anderson, P.: What is Web 2.0? - Ideas, technologies and implications for education. JISC 
Technology and Standards Watch, Feb. 2007 - at 

http: //WWW. i isc.ac.uk/media/documents/techwatch/tsw0701b. pdf (read 24 J uly 2008) 
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Virtual worlds^” 


Online virtual environment where content is created 
in the context of online game-like 3D digital 
environment to which users subscribe. 


Social tagging or 
folksonomy^® 


Ad hoc ciassification scheme (tags) that web users 
invent as they surf to categorize the data they find 
oniine. 


Peer production 
80 

news 


Web sites combining social bookmarking, biogging 
and syndication with a form of non-hierarchical, 
democratic editorial control. News stories and 
websites are submitted by users, and they 
promoted to the front page through a user- based 
ranking system. 



6.3 Technologies and standards used for peer 

PRODUCTION 

One essential factor behind "Web 2.0” deveiopment has been to enthusiasm 
to enhance the deveiopment of the new generations of web- related 
technologies and standards. The software applications are not necessary 
longer running on the user's own computer, but on the web, which seen as 
piatform. This ali is possible now when the browser technology has moved 
on to a new stage in its development - but also it is important to notify that 
the increasing bandwidth has enabled many services. Such bandwidth- 
intensive services are e.g. video distributions over the Internet. 

The new characteristics of the web services are characterised by the 
increased responsiveness and interactivity of web pages. The optimization 
of the web traffic can be achieved by exchanging small amounts of data 
with the server "behind the scenes" so that entire web pages do not have to 
be reloaded each time there is a need to fetch data from the server. This is 
intended to increase the web page's interactivity, speed, functionaiity and 
usability. 

If the "Web 2.0" amd thus also "eLearning 2.0" is becoming a reality, what 
are the future options in the technoiogy area? 

Although the views on the next stage of the Internet's evolution vary 
greatiy, many believe that emerging technologies such as the " Semantic 
Web " will transform the way the Web is used, and lead to new possibilities 
in artificial intelligence . 



see e.g. Guest, T.: Second Lives - A journey through virtual worids. Arrow Books 2007. 
see e.g. Pettenenati, M.C. - Cigognini, M.E.: Social Networking Theories and Tools to Support 
Connectivist Learning Activities. International Journal of Web-based Learning and Teaching 
Technologies, Vol. 2, Issue 3, pp. 42 - 60. 

Learning Activities. International journal of Web-based Learning and Teaching Technologies, Vol. 2, 
Issue 3, pp. 42 - 60. 
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Something what is linked also very closely into the "Web 3.0" is its relation 
with so called socio-technological values with the sense of real world. That 
will most likely also enhance the using web-based tools and technologies 
more in education and particularly in development of peer production 
practices.®^ 

Anne Fox from Experience- Based Learning Project in Denmark is describing 
the eLearning 3.0 following way: "eLearning 2.0 is about using Web 2.0 
tools in teaching. It often implies a great deal of effort on the part of the 
teacher to set up blogs, wikis, and so on as well as setting up the networks 
that their students will use to communicate with others. eLearning 3.0 will 
happen when learners take on much more of the responsibility for setting 
up their own blogs, wikis, and podcasts and for creating and nurturing their 
own networks. This implies a seismic shift in the role of the teacher to that 
of facilitator and that learners’ responsibility for their own learning becomes 
more than a slogan. Perhaps eLearning 3.0 will happen when we drop the 
"e" altogether because ICT will have become as integrated into education 
as, for example, electricity is today". 



on this discussion - see e.g. http://en.wikipedia.orq/wiki/Web 3.0 (read 24 J uly 2008) 

see Elearning 2.0 - A Stepping Stone to Elearning 3.0? http://www.checkpoint- 

elearninq.com/article/4753.html (read 24 J uly 2008) 
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This chapter discusses the hypothesis for the QMPP model - in other words 
the project's approach how to achieve and ensure quaiity in peer produced 
eLearning content. As discussed in the first chapters of this document, the 
endorsed quality approach to peer production is not seen as a standard, but 
rather as a quaiity process inciuding different stages. There are a number of 
critical aspects to be taken into consideration in these different stages. 

The quality management chalienge in eLearning content produced by peer 
production can, however, undermine the merits of this approach and 
method. The quality work methodology in peer production is at its best 
dispersed and fragmented. Often it has also been claimed, that the very 
nature of peer production is its free fiow and thus any formai mechanism 
(including the quality approach) would be drastically against the creativity 
factor. At the moment there are already a number of useful tools and 
approaches used (such as tools for peer reviews, toois for creating own 
wikis, dictionaries etc.) to ensure and improve the quality of peer produced 
eLearning content. However, it is important to emphasize that peer 
production requires also enabling and supporting structures and their 
effective management. The key issue in this project is to deveiop a holistic 
approach to peer production, which enables also the effective utilization of 
this unique method. 

The importance of peer production of eLearning content will grow especially 
in the sector of vocational education and training as well as in professional 
continuing education. Many organizations face challenges of shortening iife- 
cycie of iearning content as weil as operationai chalienges in providing 
required learning content with short lead times and lower costs. However, it 
is also understood that the learning content produced by peers (based on 
professional experience) can be more accurate and attracting than "clinicaily 
produced” learning content by external e-learning experts. 

Peer production has great potentiai in the area of vocationai education and 
training. The future workforce in Europe in many professions has not only to 
access and handle great amounts of information and knowledge, but even 
more importantly to produce various elements of information by themseives 
as an integrai part of their work. Peer production is not only a novel method 
to produce eLearning content, but it is also an approach to empower a wide 
variety of professionais to the iearning content production. Thus it has aiso 
an important democratic element in bringing the work-related learning 
content production to the actual level of users, tutors and learning 
supporters. 
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The QMPP project aims at contributing to the quality development and 
quality management of peer production of eLearning content. The essential 
work in the project is to develop a solid approach on how to support the 
quaiity management of peer- produced eLearning content, pilot the approach 
in three different VET entities, and to produce a joint tooiset for the VET 
providers of quality management of peer- produced e-learning content. 

The QMPP project itseif does not take a position of the eLearning tools used, 
but is developing and implementing a systematic process for the quality 
management of peer-produced e-learning content. 



7.1 Peer PRODUCTION CYCLE 

Within the QMPP project we have developed a metaphor for the effective 
management of quality in peer production. It includes two important 
eiements - these are the "peer production cycle" and the "supporting 
activities". 

The "peer production cycle" includes the following phases: 

♦ benchmarking - identifying of good cases and practices, identifying of 
good digital resources, identifying areas of lacking content, sharing 
learning experiences by sharing learning (b)logs etc. 

♦ creating - (shared) authoring of texts and other resources; creating 
images, audio materiais, video materials; creating wikis etc. 

♦ validating - validating content with subject matter experts, validating 
content with peers, rating the vaiidity of the content etc. 

♦ editing - sharing editing responsibilities (from proof-reading to 
translation), undertaking peer reviews, creating alternative 
navigational routes etc. 

♦ enriching - creating additional content materials, publishing individual 
works and team works, sharing or learning (b)logs, adding library 
links, social bookmarking, creating wikis etc. 

♦ updating - monitoring existing content, updating existing content, 
adding specific area content etc. 

However, as stated previousiy, it is obvious that organizations favoring peer 
production must aiso have enabling and supporting structures. These should 
include the following: 

♦ enabling policies - organizational opportunities for peer production of 
content (such as time resources aiiocated for peer production), 
management support for peer production, access to various digital 
resources to be used in content production etc. 



Education and Culture 



43 



QAfPP 



WP2 - Quality and Evaluation 
Setting the scene - 
introduction to quality in peer production of eLearning 



♦ enabling procedures - organizational support for peer production, 
guidelines for peer production and peer reviews, guidelines of 
intellectual property rights, agreement on compensation policies etc. 

♦ enabling processes - practicai support of peer production, agreed and 
supported processes and workfiows for peer production 

♦ enabling tools - joint and shared tools to be used in peer production to 
provide effective and fluent collaborative work. 

These essential elements are summarized in picture 3. 



Picture 3 




At this stage it should also be emphasized that in real life these stages are 
not linear or directly sequencial. In the creation phase there is naturally 
authoring, re-authoring etc. - thus it can rather be seen that in each phase 
there are subphases. Also it is obvious that, for instance, after the 
validating phase the creation phase is restarted and better content is 
provided for the validation. It is important to note that in the contemporary 
Internet environment these phases can be realiy fast and that the user 
communities can react realiy rapidly, if needed. 
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7.2 Exi STI NG TOOLS FOR PEER PRODUCTI ON 

It should be noted, however, that in many cases there are already existing 
good practices (and also open source tools), which have already been 
tested in various environments. This project is not aiming to develop own 
tools or toolsets. Rather the challenge is to bring together and integrate the 
various tools and approaches into a working model and solution, which can 
easily be utilized within various organizations. 

The enclosed picture 4 visualizes the various phases of the "peer production 
cycle” and points out some of the already existing practices. 



Picture 4 
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7.3 Quality CHALLENGES in peer production 

The emphasis of the QMPP project is in testing and validating through the 
pilot the real challenges and opportunities of quality management in peer 
production of eLearning. 

For the design of the experiments in piloting, it is necessary to identify the 
key issues in different phases. The following table (see table 2) summarizes 
the quality management challenges in each phase. 
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Table 2 



Peer 

production 

phase 


Key concerns and key 
questions 


Existing approaches 
and tools 


Benchmarking 


■ What content do we need 
for our learning activities? 

■ Is the required content 
already existing in a usable 
and available form? 

■ Can we access and use the 
content? 

■ Can we edit and enrich the 
content? 

■ Which obligations do we 
have with the existing 
content? 

■ How can we be sure of its 
quality? 

■ Are there (peer) references 
of the content? 


■ Search engines of the 
1 nternet 

■ Search engines of the 
"hidden web" 

■ Biogs 

■ Social bookmarking 

■ Various user groups 


Creation 


■ What content do we want 
to create? 

■ What kind of a guiding 
structure for the content do 
we provide? 

■ What type of support do we 
provide for the creators? 

■ What is the division of 
iabor between the different 
actors? 

■ How do we create the 

content? 

■ What routing of the work 
do we use? 

■ What is the timeframe of 
the content creation? 

■ How do we ensure the 

quality of the created 
content? 

■ Who can create content 
(open access vs. qualified 
peers)? 

■ How do we make sure that 

no third party IPRs are 

violated? 

■ How do we ensure the IPR 
issues? 

■ How do we ensure the 


■ (Open source) word 
processing and office 
tools 

■ Groupware tools 

■ (Open source) 

visualization toois 

■ Tools for podcasts, 
video casts etc. 

■ Media libraries 

■ Wikis and other 

structured 
environments 
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Peer 

production 

phase 


Key concerns and key 
questions 


Existing approaches 
and tools 




media richness and 

attractiveness of the 

content? 

■ Which media will we use in 
presentation and how do 
we ensure the required 
media balance? 




Validation 


■ Who should validate the 

content? 

■ Which mechanisms are we 
using in the validation (e.g. 
expert review vs. peer 
review)? 

■ How do we ensure the 

feedback of the validation 
to the creators? 

■ How do we support the 

validation work? 

■ What is the timeframe for 
validation? 


■ Routing tools 

■ Rating tools (e.g. 
giving "stars” to the 
content) 

■ Groupware tools 

■ Direct editing to the 
wikis 


Editing 


■ What kind of editing are we 
promoting? 

■ Who is entitled to edit the 
content (experts vs. all)? 

■ How do we share 

responsibilities of the 

editing work (e.g. 

voluntary division-of-labor 
vs. free access)? 

■ How do we support the 
validation work? 

■ Is the versioning a part of 

editing (e.g. making 

language versions)? 

■ How is the editing work 
validated? 


■ Dictionaries and 

glossaries (to support 
e.g. proof-reading) 

■ Groupware tools 

■ Direct editing to the 
wikis 


Enriching 


■ What kind of enriching are 
we promoting? 

■ How do we enrich the 
existing content? 

■ Who is entitled to enrich 
the content (experts vs. 
all)? 

■ How do we support the 
enriching work? 


■ Wikis 

■ Social bookmarking 

and shared 

bookmarks 

■ Digital libraries 

■ 
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Peer 

production 

phase 


Key concerns and key 
questions 


Existing approaches 
and tools 




■ How is the enriching work 
validated? 

■ Which media can be used 

in enrichment (e.g. 

podcasts, video casts etc.)? 




Updating 


■ How do we make sure that 
our content is up-to-date? 

■ How do we organize 
updating? 

■ Who is responsible for 
updating? 

■ What kind of updating are 
we promoting? 

■ Who is entitled to update 
the content (experts vs. 
all)? 

■ How is updating validated? 


■ Routing tools 

■ Alarms (e.g. based 
on calendars) 

■ Blogs 

■ Groupware 



As one can understand, this table is not complete, but it serves as a 
working metaphor for the pilots to start the planning of their work and their 
key issues. 



However, it should also be emphasized that the "inner circle" is also 
important in the quality management of peer production. Thus the issues of 
the organizational support are essential in making "quality happen". The 
enclosed table (see table 3) summarizes some of the key issues in the 
enabling and supporting structures of peer production. 
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Table 3 


Enabling and 
supporting structures 


Key concerns and key questions 


Enabling policies 


■ How do we organizationaily support peer 
production? 

■ How do we allocate time to peer production? 

■ How do we provide access to all needed 
resources, including digital resources? 

■ How do we compensate/award peer 
production? 

■ How do we ensure the appropriate approach 
to the 1 PR issues? 


Enabling procedures 


■ How do we organize the support to peer 
production? 

■ What guideiines do we provide for peer 
production in its various phases? 

■ How do we ensure required resources to 
support peer production (e.g. vaiidators of 
content)? 


Enabling processes 


■ How do we communicate of the options of 
peer production? 

■ How do we support the workfiows in peer 
production? 


Enabling tools 


■ What tools do we provide for peer 
production? 

■ Which tools do we actively support? 



The challenge of quality management in peer production is interesting. As 
peer production as a mechanism also to produce eLearning materials and 
content will grow fast, the appropriate quality mechanism can also ensure 
that peer-produced eLearning materiais and contents wiil reach wider 
audiences. 
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ANNEX: CASE STUDI ES AND THEI R DOMAI NS 



This annex describes the domain and approach described in various case 
studies. To fully understand the classification of the case studies, please get 
acquainted aiso with Chapter 7 of this report. 
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introduction to quality in peer production of eLearning 





Eureka 


1 AVANTE 


PfIegeW 


KONE 


Tax Ac 


ANITEL 


Sourcef 


Wiki 


ENRICHING 


















Potential to create freely links and linkages 






X 




X 


X 




X 


Provision of own wikis 


(X) 




X 




(X) 


X 




X 


Connnnenting of enriched content 


X 


X 


(X) 






X 


(X) 


(X) 


Rating of enriched content 


X 


X 








X 
























UPDATI NG 


















Linkages to blogs and other tools 




X 








X 


X 




Provision of alarms 










X 








Provision of routing tools 


X 




X 










X 


Provision of links to updated content 


(X) 


X 








X 


X 






















ENABLING POLICIES 


















Promotion of peer production in organization 


X 


X 




(X) 


X 








Awarding participation in peer production 


(X) 


X 




X 


X 








Provision of management support 


X 


X 






X 


X 






Provision of pedagogical support 








X 


X 


X 






Provision of technical support 


X 


X 


X 




X 


X 


X 


X 


Provision of various digital resources 






X 


X 


X 


X 


X 


X 




















ENABLING PROCEDURES 


















Given division-of-labour 




X 




X 


X 








Organizational support for peer production 


X 








X 


X 


X 




Provision of house- style guides etc. 








X 


X 


X 


X 




Provision of 1 PR-cleared digital components 










X 


X 
























ENABLING PROCESSES 


















Provision of day-to-day support for peer 
production 






X 




X 


X 


X 


X 


Agreed processes in peer production 






X 


X 


X 


X 


(X) 


X 


Agreed workflows in peer production 


X 


X 


X 


X 


X 


X 


X 


X 




















ENABLI NG TOOLS 


















Provision of standard word processing tools 


X 




(X) 


X 


X 




X 


(X) 


Provision of other standard digital tools 




X 




X 


X 


X 


X 
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